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BACKGROUND 



As part of a study to determine the relative value of various criteria 
which might be used when evaluating the worth of biology teachers, some data 
was elicited about those which professors of science education and college 
biologists felt to be most important. 

Generally, efforts to find suitable methods for evaluating teacher com- 
petence and teacher effectiveness have been disappointing. Review of these 
studies reveals much concern by investigators over which criteria should be 
applied, about who should do the evaluating, and, about the effects of sub- 
jective bias on the part of the judges involved. 

Because of concerns about the suitability of criteria and the subjective 
biases which might be introduced into the process by judges having. different 
expectations, and knowing that in spite of these limitations bioloyy teachers 
are regularly evaluated, the study attempted to reveal what various groups of 
judges engaged in an ongoing national program actually thought to be important 
and to determine ways the job category of judges related to the criteria they 
valued most. 

Even though no comprehensive studies were found regarding the evaluation 
of biology teachers, the National Association of Biology Teachers (NABT) for 
about a decade has been actively engaged in a well-organized and formal program 
of biology teacher evaluation for its Outstanding Biology Teacher Award. In 
this program (OBTA), <x teacher from each state is selected as an "Outstanding 
Biology Teacher" each year with procedures requiring that individual state 
selection committees be formed to evaluate candidates for the award. The 
composition of each state selection committee is varied, but usually consists 
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of persons representing secondary school teachers, administrators, science 
supervisors, college and industrial biologists, and professors of science 
education, 

STUDY DESIGN 

In an effort to gain some understandings of how these biology teachers 
are evaluated, the study asked each of the judges of the 1970 OBTA program 
to rate the various criteria they employed v/hen evaluating candidates for 
the award. The study involved the two hundred twenty members of the forty- 
seven state selection committees active in the 1970 program of OBTA and the 
data provided by the one hundred seventy-nine who >"eturned completed question- 
naires. The composition of judge groups is shown on Table 1. 

Assumptions made In the study were that: 

1. Criteria exist which characterize outstanding biology teachers. 

2. OBTA judges are appointed for their ability to recognize out- 
standing biology teaching and that these persons ar^ competent 
to judge biology teachers. 

3. Judges employ these criteria when making decisions about the 
suitability of candidates for the award. 

4. A 'Value hierarchy" of criteria exists, with some criteria 
being of more importance than others in evaluating candidates 
for the award. 

5. Assessment of candidates for the award is subjective and that 
individual judges value specific criteria differently. 

From related literature and materials available from the Association, 

and Incorporating suggestions from a review panel, a questionnaire containing 

one hundred eleven items which might have been used as criteria in the process 

'Q-^ developed and sent to judges to be rated according to their value to them 
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when used for the evaluation of candidates for the OBTA award. Items in the 
questionnaire included those derived from: 



- The comments by those who nominate or recommend 

- The academic qualifications of the teacher 

- The teaching and other experiences of the teacher 

- The professional activities and accomplishments of the teacher 

- The relationships of the teacher to his school and community 

- The interrelationships v/hich existed between the teacher, the 

subject, the students, and classroom organization. 



Data developed by the study were analyzed to test the following null hypo- 
theses: 

1. There is no significant difference in the ways that judges 
rate individual criteria which are used for both the ore- 
selection and final evaluation phases of biology teacher 
eval uaticn. 

2. There is no significant difference between the rating 
levels assigned by judges to the criteria used in evalua- 
tion of biology teachers. 

3. There is no significant difference in the ways that 
criteria used in the evaluation of biology teachers are 
rated between members of different judge-groups. 



The study attempted to find answers to the following questions: 



1. Who are these judges of biology teachers? What variety of 
occupational and/or educational positions do they represent? 

2. What criteria do they employ in the evaluation process? Are 
some criteria of more value to some judges than others? 

3. Does the occupational status of a judge relate to the way 
he evaluates a biology teacher? If so, in what ways? 
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Because it v;as desired to determine whether the differences between 
ratings of individual items by separate judge-groups represented differences 
due to reasons other than chance, chi-square analyses were computed for: 

1. The responses derived for each rating-level ^ within each 
separate judge-group; 

2. The responses for each ratinq-level between each separate 
judge-group; and 

3. The two responses given for each item when rated both for 
the pre-screening and .final evaluation phases. 

The degree of variance between expected and observed frequencies within 
each rating-level provided a measure of statistical significance. For 
purposes of the study, the .05 level was accepted as significant. 

SIGNIFICANT CRITERIA 

Out of the list of one hundred and eleven factors offered for 
rating, twenty-one were found to have been rated significantly high as 
usually to always important by the total grouD {Table 2). 



Although judges were asked to rate each item twice, no significant 
differences \\/ere found between the ratings assigned to those which were 
used both for preliminary screening and for the final selection phases 
of evaluation. Apparently, if significantly important at all, an item 
was of similar value throughout the total evaluation process. 

It was interesting to note that of the various groups of items, none 
of the items derived from the comments of those who nominate or recommend 
candidates, none of the items related to academic qualifications, none 
of the items related to teaching and other professional experiences, and 
none of the items related to the teacher's relationships to his school and 
community were rated significantly high by OBTA judges. Further, only one 
item was found to be significant .in the category related to the candidate's 
proftjssional activities and accomplishments, and that item related to the 
activities and accomplishments of his students. 

Generally, the twenty-one items found to be rated significantly high 
in the list could be arranged into three major areas: 

1. Items related to the teachers' intrinsic personal traits , 
including : interest and enthusiasm for biology, evidences 
of resourcefulness, adequacy of self-concept, evidences. of 
ingenuity, emotional poise and self-confidence, evidences 
of creativity, and apparent interest in self-improvement; 

2. Items related to teacher-student interrelationships , 
including ": ability to encourage self-motivation in 
students, ability to inspire self-confidence in students, 
concerns for personal involvement by students in learning 
activities, favorable perceptions by students and parents, 
ability to facilitate worthwhile student interaction, 
understandings of individual student needs, provisions 
for differing student interests and needs, and efforts 

to encourage students to develop higher level skills, 
such as advancing hypotheses and theories. 
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3. Items related to concerns for skills and proficiencies 
as a science teacher,' including: concerns for student 
understandings of essential concepts, concerns for 
students' understandings of essential science processes, 
skill in use of a variety of materials and methods, 
accomplishments of students, providing laboratory exoer- 
iences characterized by thought-provoking problems, and 
ability to develop a classroom climate which facilitates 
learning. 

Criteria found to be not significant included those related to the 
number and kinds of academic course experiences or degrees, grades received, 
location or size of school, years of teaching experience, teaching and 
managerial efficiency, participation in school, community or professional 
organizations, publications, honors or awards received, and the appearance 
of classroom and laboratory. 



EVIDENCES OF EVALUATOR BIAS 



As indicated, part of the study was to determine if evidences of bias 
on the part of judges who belonged to different occupational groups could 
be found. The data revealed that of the twenty-one items found to have been 
rated significantly hiih^ eight were found to have been rated significantly 
different between different judge-groups (Tables 3 and 4), In every one of 
the eight instances, professors of science education rated the item higher 
than college professors of biology. 

An analysis of each of these eight items follows: 

1 . Apparent Interest In Self- Improvement . Although the 
majority of each of the seven judge-groups rated this 
item high as usually or always important, only three- 
fourths of the Public School Science Supervisors group 
considered it so. This contrasted with over 97 percent 
of the Secondary School Teacher group and all of the 
Industrial Biologists. About 96 percent of Professors 
of Science Education and 88 percent of College Biologists 
rated it high . Over 6 percent of the latter qroun con- 
sidered it rarely important . 
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2, Interest and Enthusiasm for Biology , Although more than 
half of all groups rated it as either usually or always 
important, all of the Public School Administrators, all 
of the Industrial Biologists, and all of the Professors 
of Science Education responding considered it usually or 
always important to them . Only about 83 percent of Colleg e 
Sioloqists rated it high , 

3, Concerns for Student Understandings of Essential Concepts , 
Although all the Public School Administrators and ' over 95 
percent of the Professors of Science Education responding' 

■ to this item rated it usually or always important , only 
about 72 percent of the State Science Supervisors and 58 
percent of the Public School Science Supervisors rated it 
high. In fact, over 16 percent of the Public School 
Science Supervisors and approximately 8 percent of th e 
College Bio lo gists resnonding to th'is item considered it 
to be rarely important , 85'/4 percent of College Biologists 
rated it high , T 

4, Concerns for Student Understandings of Essential Science 
Processes^ Although the majority considered this item to 
be usually or always important, differences existed between 
some groups. In this instance, all of the Professors of 
Science Education rated it high , while only 58,4 percent of 
the Public School Science Suoervisors rated it thus. Almost 
17 percent of the latter group considered this ''tem to be 
rarely important, 78 nercent of College Biologists rated 
this item high . 

5, Ability to Inspire Self-Confidence in Students , Responses 
to this item were fairly diverse and ranged from 100 percent 
of responses as either usual ly-to-always important for the 
State Science Supervisor group, to only 50 percent of the 
Public School Administrators rating it high. Several 
respondents rated the item either rarely important or not 
important to them in the evaluation process, 80 percent 

of College Biologists and 82 percent of Professors of 
Science Education rated it higli , 

6, Activities and Accomplishments of Students , Analysis of 
data for this" item revealed that although all of the Public 
School Administrators and Industrial Biologists rated it 
either usually or always important, this feeling was not 
shared by several of the other grouns. In fact, only about 
47 percent of the Secondary School Teachers and a little 
less than 60 I'orcent of the Publ ic School Science Super- 
visors and"l/yi lege Biologists rated' it high. Of significance 
werq/tPie rerponses which rated this item as rarely Important 
or not imp o rtant . These included Secondary School Teachers 
with 12,5 percent , Public School Science Supervisors with 
16.7 percent. College Biologists with 10,7 percent , and 
Professors of Science Education with 8,3 percent , 79 percent 
of Professors of Science Education rated this item high > 
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7. Emotional Poise and Self^Confidence . Analysis of the data 
for this group reveciled th^t of those responding to the 
item as an item of importance to them, all judges- in the 
Industrial Biologist group rated it either usually or 
always important. This contrasted with Public School 
Science Supervisors and State Science Suoervisors whose 
responses in these categories amounted to about 65 percent 
each. 17.7 percent of the Public School Science Supervisors 
and 10.9 percent of the College Biologists thought this 

■ trait to be rcjrely Important to them . About 91 percent" of 
Professors of Science^ Education and 78 percent of College 
Biologists rated this item high . 

8. Adequacy of Self -Concent . Contrasts between the ratings 
assigned by various judge-groups were particularly noticeable 
for this item. Even though the majority of all groups rated 
this item high as usually or always important, 25 oercent 

of the Public School Science Supervisors considered adequacy 
of the teachers^ self-concent to be rarely important, along 
with more than 12 percent of the Public School Administrators. 
Only approximately 50 nercent of the College Biologists rated 

■ the j' tern 'high, obth Professors of Science Education and 
College Biologists rated this item lowest with 73.7 nercent 
and 51.4 percent respectively. . 



SUMMARY AND COiNCLUSinNS 

Throughout the course of the study, the intent was to discover something 
about the evaluation of biology teachers and not to assess the worth of the 
Outstanding Biology Teacher Award program. The OBTA program was used because 
it offered an excellent opportunity to collect data about what judges of 
biology teachers value on an unusually comprehensive scale. The investigator 
does not wish his conclusions to be -interpreted as judgements of the program, 
although he is impressed with it as a model for teacher evaluation by a pro- 
fessional group. 

The study attempted: (1) to establish the criteria that judges in the 
Outstanding Biology Teacher Award program used when evaluating biology teachers; 
(2) to identify the various types of their occupations; (3) to establish whether 
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or not SDecific criteria were valued significantly different; and (4) to 
determine if occupational status of judges were significantly related to 
the way they rated specific criteria. 

To facilitate brevity, conclusions are organized under each of the 
three null hypotheses established for the study: 

Null Hynothesi 2 1 : 

There is no significant difference in the ways that judges 
rate individual criteria which are used for both the pre- 
selection and final evaluation phases of biology teacher 
evaluation. 

Analysis oi-' tiie chi-square levels found for the combined ratings of 
all judges revealed that no significant differences existed between the 
ratings given to items when used for pre-selection and the ratings given 
to the same items when used for the final evaluation of candidates for 
the Outstanding Biology Teacher Award. Because the criteria were not 
rated significantly different, it appeared that judges made no important 
distinction between criteria which they used to pre-screen candidates 
and those they used to eliminate finalists. Thus, Null Hypothesis 1 was 
accepted. 

Null Hypothesis 2 ; 

There is no significant di f f erence between rat i ng 1 evel s 
assigned to juHges to the criteria used in evaluation of 
biology teachers. 

Analysis of the chi-square levels derived for the ratings given to 
each of the one hundred eleven items revealed there were twenty-one items 
(Table 2), which possessed high rating levels that differed from what might 
have occurred by chance at the .05 level of significance or better. 
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Of the twenty-one items which judges rated high, seven related to 
factors associated with the teachers* intrinsic personal characteristics, 
eight related to factors of teacher-student interactions and six related 
to skills and proficiencies as a science teacher. Factors which were 
rated lowest related to the teacher's academic background and preparation, 
his teaching experiences and responsibilities, and to his professional 
activities and accomplishments. Apparently, these latter factors were not 
as important to these judges. Because some significant differences were 
found, Null Hypothesis 2 was rejected. 

Nul 1 Hypothesis 3 : 

There is no significant difference in the ways that criteria 
used in the evaluation of biology teachers are rated between_ 
members of the different judge-groups-. 

Analysis of chi-square derivations for ratings given to each of the 
one hundred eleven items of the questionnaire revealed that eight items were 
rated significantly different by different judge-groups. Thus, it was pos- 
sible to say that the occupation status of a judge is related to the way 
he evaluates a biolpgy teacher, and that the criteria he used. were applied 
according to his expectations of the teacher's role. Therefore, Null Hypo- 
thesis 3 could also be rejected. 

In conclusion, regarding evaluation of biology teachers, it is possible 
to say that specific criteria do exist which are significant to competent ■ 
judges. And, if significant at all, these criteria are important generally, 
rather than being selectively applicable for making either preliminary or 
eventual decisions about the teacher's worth. Also, concerns -for evaluatqr 
bias are real concerns, that various judges do value specific criteria dif- 
ferently, and that Professors of Science Education and College Biologists 
do differ somewhat in how they value specific criteria. 
O 
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Table 1 * 

Occupational Groups Represented in the 
1970 QBTA State Selection Coimiittee s 

4 

Number on 







Committees 


Replied 


1. 


College Biologists 


56 


50 


2. 


Secondary School Teachers 


58 


41 


3. 


College Professors of Science Education 


. 28 


25 


4. 


State Science Supervisors 


21 


18 


5, 


Secondary School Administrators 


17 


16 


6. 


Industrial Biologists 


21 


13 


7. 


Local Science Supervisors 


15 


12 


8. 


Other: Dentists 

Director, Outdoor Education 
Graduate Student, Education 


2 
1 
1 


2 

1 
1 




TOTAL 


220 


179 
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Table 1 



Significant Criteria Employed by Members of 1970 Selection Committees 
in Making Decisions About Candidates for the OBTA Award" 



Item Significance 

Interest and enthusiasm for biology « .001 

Ability to encourage self-motivation in students .001 

Concerns for student understandings of essential concepts .01 

Ability to inspire self-confidence in students ,01 

Concerns for student understandings of essential science 

processes .01 

Evidences of resourcefulness .01 

Adequacy of self-concept .01 

Concerns for personal involvement of student in 

learning activities .01 

Evidences of ingenuity .01 

Emotional poise and self-confidence o .05 

Evidences of creativity ' .05 

Apparent interest in self-imnrovement , .05 

Habits of dress, voice, mannerisms, speech .05 

Activities and accomplishments of students .05 

Provisions for differing student interest and abilities .05 

Laboratory experiences characterized by thought-provoking 

problems ; .05 

Efforts to encourage student development of hypotheses and 

theories .05 

Favorable perceptions by students and parents .05 

Facilitates worthwhile student interaction , .05 

Ability to develop a classroom climate conducive to learning ... .05 

Perceptions of individual student needs .05 
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TABLE 4 

FACTORS FOUND TO DIFFER SIGNIFICANTLY 
BETWEEN VARIOUS JUDGE--GROUPS 



Rating Level Perce.' 

Judge-Group V " 2 ""^3 ? 5 Ratin-i tiie 



Apparent Interest 1n Self-Improvement 

Professors of a 0 v 0 1 4 19 

science education b 0.0 0.0 4.2 16.7 79./' 95. 



College 3 0 3 12 31 

biologists 6.1 0.0 6.1 24.5 63.3 87, B 

Emotional Poise and Self-Confidence 

Professors of 1 1 0 8 12 

science education 4.5 4.5 0.0 36.4 S^.S 90.2 

College 1 ' 4 5 21 15 

biologists 2*2 8.7 10.9 45.7 32.6 78.3 

Concerns for Student Understandings 
of Essential Concepts 

Professors of 0 0 1 5 17 

science education 0.0 0,0 4.3 21.7 73.9 95.6 

College 2 2 3 13 28 

biologists 4.2 4.2 6.3 27.1 58.3 85.4 

Interest and Enthusiasm for Biology 

Professors of 0 0 0 3 21 

science education . 0.0 0.0 0.0 12.5 87.5 100.0 

College 2 1 5 11 30 

biologists 4.1 2.0 10.2 22.4 61.2 83.6 
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TABLE 4 (continued) 



Judge-Group Rating Level i'ercent 





V' 


2 


3 


4 


5 


Rating Uie 
items High 


Activities and Accomplishments of Students 













Professors of a 
science education b 


0 

0.0 


2 

8.3 


3 

12.5 


7 

29.2 


12 

50.0 


79.2 


College 
biologists 


2 

4.3 


3 

6,4 


14 

29.8 


16 

34.0 


1? 


59.5 


Ability to Inspire Self-Confidence 
in Students 














Professors of 
science education 


0 

0.0 


2 

8.7 


2 

8.7 


3 

13.0 


16 

69 . n 


82.6 


College 
biologists 


0 

0.0 


3 . 


6 

ion 


16 

Q/1 Q 
O'f • O 


21 




Adequacy of Self-Concept 














Professors of 
science education 


0 

0.0 


1 

5.3 


4 

21.1 


6 

31.6 


8 

42.1 


73.7 


College 
biologists 


1 

2.9 


1 

2.9 


15 

42.9 


13 
37.1 


5 

14.3 


. 51.4 


Concerns for Student Understandings 
of Essential Science Processes 














Professors of 
science education 


0 

0.0 


0 

0.0 


0 

0.0 


6 

26.1 


17 

73.9 


100. 0 


College 
biologists 


2 

4.3 


2 

4.3 


6 

13.0 


11 

23.9 


25 

54.3 


78.2 



^Number of responses 
'^Percent of responses 




1 = Not Important; 2 = Rarely Important; 3 = Sometimes Important; 
I 4 = Usually Important; 5 = Always Important,' 



